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ABSTRACT 

Proficiency Verification Systems (PVS) is a new 
concept in providing management information about local achievement 
in basic skills. The program includes a network of assessment and 
reporting components which can be combined in varied ways to generate 
proficiency information about individual pupils and groups, for 
teachers, principals, and school administrators, A key feature is the 
capability to summarize reports according to use of specific 
instructional programs, and to prpvide background reaalts obtained 
f^om all users of the system, initial field testing and revisions 
hagan in 1975^76 and continues with over 20,000 pupils in three 
states, (Author) 
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ABSTRACT 

Proficiency Verification Systems (PVS) is a new concept in providing 
management information about local achievement in basic skills* The program 
includes a network of assessment and reporting components which can be 
combined in varied ways to generate proficiency information about rndividual 
pupils and groups » for teachers, princlpais, and school administrators* A 
key feature is the capabirrty to summarize reports according to use of 
specific instructional programs, and to provide background results obtained 
from all users of the system. 

Initial field testing and revisibrrs began in 1975^76 and continues with 
over 20j000 pupils In three states. 



INTRODUCTION 

Proficiency Verification Systems (PVS) Is an sssessment and reporting 
program whlGh teachers and administrators can use to show what skflls 
pupMs have developed successfully during the school year. The program 
integrates a variety of skill I nventori es wi th d I f f erent ways of reporting 
results* It was developed at SWRL.from more than ten years of research on 
cr I teri on^referenced nieasurefnent to provide comprehensive information for 
bettpr Jirknowl edgement of instruct lonal accompl fshments in individual 
classroomSi schools^ and entire school systems* Local school districts 
choose from among the PVS componen ts the specific combination they wish to 
use, and have several options for deciding how results are to be reported* 
PVS skills inventories are simple to administer and require no scoring by 
teachers. * 

Severa l character 1 st lics of the PVS program are unique among systems 
that are currently available for organizing informatiom about pupil 
accomplishments. 

• School districts can customtze their own information system by 
choosing PVS components and reports to satisfy unique local needs* 

e Remote data processing Ps unnecessary for many PVS components* 
Information for week-by-^week classroom instruction Is generated 
and organized exclusively at the classroaTi level to elimiriate 
delays inherent In computer process ing and to ensure that this 
information v/i 1 1 afways be available for classroom decisionmaking 
where and when It is needed most, Only when broade.r^prof 1 1 es are 
desired to periodically provide informatFon pertinent to the 
instructional accompVr shmcnts of a set of schools or the district 
as a whole Is remote data processing incorporated into PVS operations 
and procedures. A 



# PVS reporting systems are able to tie Inventory resiilts directly 
to specific programs chat local schools are using nsgularly for 
classroom instruction* 

m National reports provrde local schools with comparison data from 
all other schools and districts using PVS with due regard for the 
protection of personal and i ns 1 1 tut I ona I privacy, 

m Separate reports are prov 1 ded for cl ass room teachers, principaISi 
and district administrators. These reports describe instructional 
accomp I t shments ranging from overall achtevement by an entire 
district to diagnostic profiles of Individual pupns. 

m In operation* PVS allows loca T school d i st ri cts to main tal n their 
own data bases and procedures for hcM skiSU Inventories are 
administered and reports are used. Districts can maintain 
maximum con f I den 1 1 a U ty ' f or teachers and pupils by assigning 
their own data processing eodes to materials that are returned 
to SWRL for scoring and processing. Local schools have complete 
control over school, teacher » and pup!) identification within the 
sys tern. 

The R&D associated with PVS promises to deliver clearly In the 1970' s 
what was glimpsed obscurely by researchers In the igfiO's as -'computer 
managed instruction,'' "criterion re f erenced tes 1 1 ng i " and 'Mnstruct ronal 
task analysis." The products that comprise a Proficiency Verification 
Systenj provide for schools the Immediate resources that CMI , CRT, and ITA 
separatGly wi 1 1 never provide^-a means of d i rec 1 1 y re 1 at i ng their Instruc- 
tional objcctlvcSp instructional materialsi and standardized tests, with 



al lowunctis fur cholc^j uniqueness, and commonalfty at the class, school ^ 
and district leve I , 

PVS RKU bcinrn s«^*iiu rt* 1 a t i oiish i p to the lltertiturt* nnd ncLlvtty boini] 
generated at such locntions ab LRDC, UCLA, the University of Wisconsin 
R&D center and facuJtyp Florida State Un ivers 1 ty , ETS, NAEP, UC Santa 
Barbara, University of Minnesota, University of Massachusetts ^ and so dn; 
school districts such as Los Angeles^ Dallas, Miami; state departments 
such as California,, Michigan, Florida, New York; publishing firms such as 
Harcuurt, McGraw^fi i 1 1 , SRA, Houghton MjffHn; and cOTputer firms that have 
now dwl,ndled essentially to Control Data, HewMtt Packard, and National 
Coniputer Systems. 

PVS RSD as such began at f-^L under NIE contract to design and initiate 
the development of Proficiency Verification Systems in Mathematics and in 
Reading. The objective is to generate sets of pupi! inventories In mathe^ 
matics and reading that have fi rm ground Ing In e>c f s t i ng I ns t rue t lona 1 programs 
in elementary schools, and to use these inventories as the nucleus of a system 
that provides local school districts a means of collecting and reporting 
information to show local instructional accomplishments at various levels of 
school d i s t r f ct organ I za t i on . To date, a fully operational system in 
mathematics has been duvyloped (Buichanan, 1976) and Is currently being used 
with over 20^000 pupils in four states. The groundwork has also been 
established for a comparable system In reading (Fi eger^Kol Imann ^ 1977.K 

Although PVS development began with the NIE procurementj the "rapid'' 
development progress of PVS wais enabled by programmatic R&O at SWRL over 
the past ten years. The scoring and report I ng structure (Hllazzo, 1976)i 

e 



whicfi curruntiy uses the NCS Century Model 7010 Optical Mark Readers has 
benefltled from SWRL's earlier research on systems to provide cpjnprehen-' 
sive instructional manacjGment inforniat ion (Behr and Gibbs, 1975)* The 
Fortran IV software for PVS uses sane of the same routines for compiling 
data as the Quarity Assurance program (Wolfe, 1976) which SWRL matntains 
to assess the effects of its own instructional products. The substantive 
structure of PVS Inventories, which incorporates the basic content donia ins 
of prevailing mathematics programs within a single Item pool, bears some 
relationship to an earlier SWRL study (Hansohp 1977) that determined and 
contrasted the effects of different programs for teaching reading readiness 
in kindergarten. To establish the needed structure for PVS, it has been 
necessary to obtain robust techniques for analyzing instructional programs 
across several grade levels. These techniques evolved from computer-ass i sted ^ 
analytical procedures, that were developed between 1973 and 1975 for SWRL's 
Learning Mastery Systems, The latter were criterion-referenced systems for 
assessment and remediation that were developed by SWRL to accompany specific, 
commercial programs in matKsmatics and reading, 

SWRL's RSD in PVS is part of a line of inquiry related to "schooling 
and learning.*' PVS activities are designed to treat "school i ng and learning'' 
as a joint pimnomcnon that can be observed directly, without regard to 
preconceived notions of how the learning act occurs or what pupils need tO 
accomplish* The intent Is to make objective descriptions of what pup! Is are 
expected to learn based on the substantive content of their formfii! learning 
eKperlences, and to structure the flow of information about actual ski 11 
development accordingly* What Is Implied by proceeding in this way Is 
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that '^schooling" is a% much a social phGnomcnon, thnt is amenable to bona 
fida scientific inquiry, as it is an Institutionalized service in need of 
evaluation^ 

PROGRAM COMPONENTS FOR GENERATING INFORMATION 

The core of the PVS program consists of three types of ski 11 
inventories designed for use at specific points during the school year. 
It also includes a straightforward system used entirely v/ithin the 
classroom for collecting and organizing small bits of Information about, 
the progress of each pupil as a more-*or- less continuous record^keep i ng 
process maintained by the teacher during the school year. Each of the 
skills Inventories can be extended by special components that will be 
available in PVS (1978), depending on the needs of local schools and their 
ability to commit more pupil and teacher time to the maintenance of a more 
comprehensive information system. 

Proficiency Skills invontjDrjy 

The primary tool is the Proficiency Skills Inventory (PSl) that is 
used typically at the end of the school year. This mventory Is a 
comprehensive set of Items represent: i ng skills that ar^i regularly developed 
across the country at particular grade levels. Tha set of items for each 
grade level Is obtained and regularly updated from a methodical and detailed 
analysis of mathematics programs, standardized achievement tests, and 
curri cu lum gui des that are widely used for Instruction In the elementary 
school . 
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The oKtent CO which serious analysis of mathematics programs Is 
0 cnrnarstone for dcveloptrui ^kill invcn lories is unique to the PVS 
proqrntn. Most s Inndnrd I ?cmJ nch i c^vonuMi I icsls finclini^ I hei r nch leveincnl 
norm^ to gross dlfferenGes in Instructional content across grade levels* 
but there Is no commitment to any sort of comprehensive representation 
of the wide variety of skills developed within a single grade level of 
instruction. In PVS, inventory items are chosen on the basis of what 
pupils are likely to have the opportunity to learn. The main criterion 
for the Inclusion of an Inventory item is whether It resembles, to any 
significant degree, the kinds of instructional items that pupils are 
expected to solve in one or more major mathematics programs. Items are 
not rejected in PVS simply because they are too easy or too difficult. 
If an I tern is like the content of current instructional materials, then 
It belongs In a PVS Inventory, This I nstruct 1 on^based rationale for item 
suitability varies somewhat from common practica for constructing standardized 
tests, where Individual Items must eKhlblt certain characteristics in use for 
them to contribute to reliability of the overall test* In such a settingi 
the suitability of items must be based partly on factors that have little 
direct relationship with actual Instruction and opportunity to learnt 

Levels (1; 2, of the PS! correspond Indirectly to conventional 

grade levels* since age^graded patterns for school organization continue to 
prevail across the country* However* It Is not necessary that a particular 
luvel of the PSI be used eKcluslvely with pupils at the correspond Ing grade 
level* The deslgnatod level of mater 1 a 1 s that pupils regularly use during 
instruction Is the keyj regardless of their assigned grade in school, since 
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i ttiilis at each ievul of the PSI are deterniined by the organization of 
nia terialb In exisLinq rnathenia t i cs pronrams. What this means, for example,- 
ib that usu of Levc ! 2 of the PSI with ciracJe 3 pupils Is perfectly proper 
if these pupils regularly use a textbook Intended for normal use In grade 2 

In operation, each level of the PSI is printed in four forms (Aj B, C, 
and D) , Dividing the total item array Into multiple forms Is neeessiry 
since too many items are included in the overall Inventory for any one 
student to take vMittiln a reasonable amount of classroom time* In actual 
administration, all four forms are given= in each classroom in a balanced 
way so that all items on the PSI are represented for each group of pupils 
taking the test. In the school and district overall, this mean that each 
item will have been given to a large number of pupils^ although not all 
pupils will have responded to every item in the total array of PSI items. 

Some items on the PSI represent skills that are more Important than 
others for pupils to develop* From an analysis of mathematics programs, 
currtculum gyides, and standardized testing programs, It is readily apparen 
that not ail skills taught in the elementary schools have the same level of 
importance* At eacti grade level there are a number of minor skills that 
rece i ve re I a t i ve I y little i ns t ruct i ona 1 t i me * The most i mportan t skills 
are benchmarks to the entire mathematics sequence since they are closely 
interrelated with skills from preceding grade levels and with skills that 
are programmed to follow at a much later tlmei 

At each level, each Form of the PGI Is divided Into two parts which 
can be administered separately on different days* Part One contains all 
of the items related to benchmark skills, which are mainly In the areas 
of number recognition, computation facts, computation algor I thms, and 
verbal problenis. All of the items in Part One are the same on all forms 

10 



-8- 



of l\\u irivtMit<iry 4U QQC\\ iGvel. By designing the inventory in this way» 
each pupil rcs|)onds to enough I teiiih raprosenting the benchmark skills so 

thnt tuochars can rccclvu dincirujstic prurilos o( per lOnnancu by iiulividUiil 

pupiib and by Lhu entire class* Only on Part Two, representing skills 

that are not of prime importance at each grade level , are items varied 

on different forms of the inventory* This way, all of the material ^ 

typically developed at a particular grade level can be represented in a 

single inventory, and, at the same time, diagnostic Informatiun can be 

obtained for each pupil on benchmark ski 1 1 s. 

For each pupil , the PSI consists of 70^00 items, and usual ly requi res 
tv^o class periods of about kO minutes each for administration. More time 
may be used If necessary since Inventory results are not intended for 
interpretation against a "normaP* sample of pupils who have taken It 
previously under standardized conditions* This way, the PS! gives a more 
complete representation of what pupils are able to do In relatior' to what 
is expected and what skills they are likely to have had the opportunity 
to learn* 

Current versions of the PSi consist entirely of mu 1 t i p 1 e^cho i ce 
items. Skills requiring pupil reactions which cannot be adapted very well 
to a niu 1 1 i p le^choi ce format are reserved for special PVS inventories*^^'- 
Tliese skills includu nuiDeral writing, geometric constructions, open-ended 
problems with a variety of possible solutions, and so on. For Levels I = 3i 
pupils mark answcri^ directly on a scannable inventory booklet* Separate^ 
scannable answer sheets are used with inventories for Levels 4-6* 



-Those fnvontorfos are still under development and will not be available 
until 1973. ' 
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Beginning Skills Inventory 

PVS Includes an inventory for obtaining baseline and readiness 
information, called the Beginning Skills Inventory (BSl), designed for 
use near the start of the school year. Each BSl Includes* (1) major 
skills that were scheduled for development during the previous year and 
(2) extensions of these skills that are likely to take place durrng the 
coming year. The items included are obtained from PVS analysis of widely- 
used programs, curriculum guides^ and standardized tests using the procedures 
described earlier for the Prof lei ency Ski 1 1 s Inventory . 

Beginning Skills Inventories are available for Levels 1-6 corresponding 
to age-grade levels of conventional school organization in the same ways 
described for the Proficiency Skills Inventory^ the level of the BSI that 
Is appropriate for a particular group of pupils depends on the designated 
level of instructional materials they are using, and not necessarily, on 
their age or grade in school. 

The items of the BSI at each level focus on a combination of most of 
the same benchmark skills represented In Part One of the Proficiency Skills 
Inventory* Some skills on the BSI parallel benchmark skills from the 
Proficiency Skills Inventory for the previous levels and some items represent 
extensions of benchmark skills In the current level. The BSI is not a pre-- 
test. it does not Include Items that are totally different from instruction 
the pupil should have had previously. Some skill extensions are a part of 
the BSI but they consist of items or problems that look much like material 
studied before and can be solved with minimal transfer from previous 
learning* For example, In Level 3, the, BSI includes items on 2-digit and 

.... y , ... ........... 
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3"cl ig 1 1 add i t i on , a 1 though most pup Ms will have had on 1 y Z-^d ig 1 1 add i t ion 
during the previous year. On the other hand, the BSI for Level 3 does 
not incl udc I terns on the mul t i pi ica 1 1 on a Igor 1 thm, because th is content 
is nuL Introduced In major mathematics programs until mi dway through 
grade 3 and would be totally new for pupils at the beginning of the school 
year. The key for the BSI is what sl< 1 1 1 s pup I 1 s have had 'an opport unl ty 
to learn prev iousl y / and what they a re 1 1 ke I y to know already by way of 
s imp 1 e , d j rect general I zat Ion of skill learning from the prev ious year * 
: Each level of the Beg i nn ing Ski 1 1 s Inventory has only one form, so 
all pupils are given the same i terns # Skills related to Part 2 of the 
Prof iciency Ski I Is Inventory are only represented i nd i rect 1 y , i f at a 1 1 j 
on the BSI, since information about such ski lis would have limited useful- 
ness for teachers and /jdmlnlstrators In developing the! r long range plans 
for instruction at the beginning of the school year^ 

The BSI for each level consists of 30'^0 multiple-choice ItemSj and 
usually takes one class period of 35^^0 minutes to administer . More time 
may be taken, If necessary, since test conditions are not Intended to be 
standardized. All of the items on the Beginning Skills Inventory have 
mu 1 t I p le"choi ce formats* Pupi Is mark answers di rectly on the inventory 
booklet for Levels N3- For Levels ^=.65 they use a separate answer sheet* 

Mi d^Year I nventory 

The Mid-=Year Inventory (MY I ) Is designed to be used around the last 
two weeks In January* It is a set of items that show what pupils have 
accompl i shed by the time they complete the first half of the schpol year. 
Development of the MYI is based. on the same analytical techniques that 
are involved in the Proficiency Skills Inventory. 



A Mid-Year Inventory is provided for Levels 1*6, corresponding to 
agc^grado levels in conventional school organ izat ions in the same way as 
the Prof I ci ency Ski 1 Is Inventory and I n 1 1 1 al Ski 1 1 s Inventory, The criteria 
for selecting the appropriate leveT of the MYI foh any group of pupils is 
the levelof insiructi onal mater ials they are using regularly, not their 

■ grade level in schooK 

Within each level of the MYI, items focus on the same benchmark ski 1 Is 
represerited in Part One of the Proficiency Ski I'ls Inventory* ^ The inventory 
Itself Is divided into sections according to the major benchmark ski 1 1 s and 
sub-ski^Ils* Although the Inventory Is Intended for use as a mid-year progress 
report, the Jtems cover projected instruction for the entire year. This way, 

- . teachers may use the MYI In a variety of ways. It Is not necessary, for 

eKample, to use a textbook in chronological order. The teacher should only 
administer the sections on the MYI representing instruction on benchmark 
ski 1 1 s that pup i 1 s have a 1 ready completed during the f I rst hal f of the 
school year. Sections of the inventory on skills that are direct extensions 
of instruction during the first half of the year may also be administered, 
but teachers are not encouraged to give sections of the MYI dealing with 
skills or concepts that will be introduced for the first time later in the 

. • year. . ; - 

Each level of the MYI has only one form, so Items will be the same for 
all pupils. Ski II s that are 1 1 ke Part Two of the Proficiency Ski 1 1 s I nventory 
are represented on ly Ind i rect ly In the MYI, 

The Mld^Year Inventory at each level consists of 30 to mul t Iple-choice ^ 
items, and usua 11 y requi res one class period of 35^^0 mi nutes- for admin i strat lonv — - 
More time may be taken If It I s needed, s Ince the MYI Is intended to be more / 
; . ;; representative of what pupils can do in relationship to what has been 

presented during I ns truct I on , than what pupils can do In a rigidly- - 
con L rol led t i me frame , 14^ . \ 

EKLC 
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Learning Mastery Profiles 

For week-by'week management of i nstruct i on , PVS provides a series of . 

■ ■ ■ . 

Learning Mastery Prof i les developed specifically to accompany major, 
comme^rci a 1 programs In mathematics* Basically, a Learning Mastery Profile 
assists the teacher by identifying brief skills checks from pupils* work 
In regular, day- to-day I nst ruct t on , and In organizing this information as 
a record of pupil progress on a conti nuous bas i s during the yoar. The basic 
component 1^ simple to use/ since it makes use of the instructional materials 
that already are in use in the classroom* Separate materials for assessment 
and practice are available from SWRL in kits called "Learning Center Resources 
These materials are not part of the regular PVS program, but were designed for 
coordinate use with the Learning Mastery Prof i 1 es* 

The major component is a separate Prof I le sheet, for maintaining a record 
of performances for each pupil during the year. The Ski 1 1 s Checks are shown 
here alongside spaces for designating how well pupils have done. In addition, 
spaces are provided for organizing other Important Information about pupils* 
performances Including: chapter tests and level tests from the regular 
mathematics program, the PVS Beginning Ski 1 1 s I nventory , M!d=Year Inventory, 
and Proficiancy Skills Inventory, guidelines for pacing i nstruct i on during 
the year, and so on* 

Span I sh- Language j/er 5 Ions of PVS I nven tor ies • 

All PVS Inventories in Mathemat 1 cs have English and Spanish versions* 
At grades I, 2 and 3s where pupils mark directly on the PVS booklet, there 
are para 1 le I sets of booklets In both languages. In addition, at the higher 
grade levels, the standard answer sheet contains directions In both Spanish 
and Eng 1 i sh. 

. . ■ • 15 ■ 



\ PVS REPORTS' 

During the school year* PVS provides a series of reports with 
Information channeled to key personnel at d i f f erent 1 evel s of school 
di s tr i Gt organi zati on* Reports for ind Ivi dua 1 pupj Is and classrooms 
are sent to teachers; reports for schools are sent fto the var i ous pr 1 nc i pa 1 s ; 
and reports for the district are sent to a designated PVS coordi nator within . 
the district administrative staff. 

PVS has an Important capabfllty to rmport on how pupMs in dtfferBnt 
Instructional programs performed on all Items In PVS components and on Just 
those items that represent the programs they are using* This kind of report 
Is possible because of the way PVS inventories are developed. All Items on 
PVS Inventories come from an analysis of major instructional programs, 
curricuItiM guides, ^nd assessment programs* Each Item In a PVS_component ... . 

Is "tagged*' wl th a record of the resource where It originated* Some items 

=^ . . ... 

are common to a number of instructional programs and other resources* Others 

. . . . . ■ : ■ . ; . .. . _ - .- 

represent a part of Instruction In only one program, 

Characteri St i GS of PVS reports vary somewhat wI th each of the skM Is ^ 
inventories* The way data are organized and the iklnds of data Included In 
the report are not the same, because the results of different PVS inventories 
used during the year are not 1 ikely to be used In the same way. Reports from 
the end-^of-year Proficiency Skills Inventory, for examples, are ^very likely to 
be used in overall consideration of hoiv well an instructional program has met 
district needs during the sGhool year, so these reports Include a breakdown 
of pupil proficiencies by instructional pr^rams within the district and 
across the national population of PVS users* On the other hand, reports 
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from the Beginning Skills Inventory are more likely to be used In long- 
range pianning for the schoQl yearj and a breakdown of results by program 
would not meet the same kinds of information needs that are associated ^ 
with the Prof iciency Ski I Is Inventory. 

The data provided In PVS reports are Intended to satisfy dual needs 
for eval uat I on of Instruct ion that has a I ready taken pi ace and for long- 
range, diagnostic planning. In most cases, the reports for the Prof iciency 
Ski 1 1 s I nventory show the full range of reporting character I st Ics In PVS. 
For all other components, the characteristics 0 PVS reports are the same ^ 
essentially, but they arte not as eKtens i ve. 

Proficiency Ski 1 Is Inventory . . 

For the Prof icfency Ski 1 1 s Inventory, there are four basic reports. 
Other reports may also be generated , depending on the arrangements made 
with Indlvidua! school districts^ but they are not part of the regular set 
of reports and may requls'e additional information about pupilSj district 
programs, and the local community, . 

The first report V cal led the PVS National Report ^ shows how the en t i re 
population of pupi Is In PVS did on the PSI items. This: report is in four 
parts that examine successively smal ler units of the results as followsr 

PART 1: RESULTS ON OVERALL INVENTORY 

PART 2: RESULTS ON BENCHMARK SKILLS 

PART 3: RESULTS ON ALL SKI LL AREAS (BENCHMARK AND NONBENCHMARK) 
PART hi RESULTS ON INDIVIDUAL ITEMS. 
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InParts I, 2, anrt 3 i the results are shown as follows: 
AVERAGE SCORE BY ALL PUPILS , 
AVERAGE SCORE BY PUPILS IN PROGRAM x'' 
. ' :0H ALL ITEMS - 

:0U ITEMS FROM PROGRAM X 
AVERAGE SCORE BY PUPILS IN PROGRAM Y 
:0N ALL ITEMS 

:0N ITEMS FROM PROGRAM Y : 
\n Psft the results for each 1 1 am are broken dc^n as fonowsi 
PERCENT OF PUP! LS ANSWERING CORRECTLY 

I ALL PUPILS IN PVS 

:PUPILS IN PROGRAM X 

I PUPILS IN PROGRAM Y 
The second type of report, called the PVS District Report, is structured 
in four parts Just as the Nat i ona T Report . This report also has the same 
breakdown of information as the National Reportj but ft I s fo jRS ed _ on 
programs used within the d i strj^ct and the performance of_ df strict pupils ^ 
The third type of report is for principals within the district* Tha 
PVS School Report comes in two parts shwing how pup! Is wi thin a specific 
school performed on the test overall and on benchmark skills* Within each 
part the results are reported for all pupils within the school, and for 
pupils in each I nst ruct i ona 1 pr^ram that teachers are using. A separate 
report showing pupil performances on each Item is also provided routinely 
in the PVS School Reports. 



^ In actual reports, programs will be identified by ah abbreviated 
program name (e*g. , HM/Mathemat I cs f or Individual Achjevement). A key 
to these names will be given at the beginning of the reports. 



\ Th- fo^ type of report is the PVS C lasaroom Report for each 
teacher admf nister ing the Prof iciency Ski ) 1 s Invanto This report 
comes in three parts w1 th information focused on individual pupi Is. The 
first part shows how individual pup! Is performed on each of the benchmark 
skill areas, and their overall score on other areas of the PSI that reflect 
instruction in their regular program. The second part of the report shows 
what percent of pupils In the entire class answered correctly on each 
benchmark items which are described 5n terms of the i r mathemat i cs objectives* 
The third part nsts those pupils who scored less than 501 of the items 
correct on any particular benchmark skill area, 

Reports for the Beginning Skills Inventory 

The reports for the Beginning Ski 1 Is Inventory are less extensive than 
reports for the PS I • Four reports are generated Just as with the PSI , but ^ 
the internal structure of each report Is a 11 tt le d i ff erent . The National 
Report for the Beginning Sk i 1 1 s I nven tory has two parts as follows: 

PART 1: RESULTS ON OVERALL INVENTORY AND ON BENCHMARK SKILL AREAS 

PART 2r RESULTS ON INDiVIDUAL ITEMS 
No breakdown by instructional program is given for BSI results, since ft 
would have 1 1 tt le, clear ut i 1 I ty for planning at the beginning of the 
school year. For each item or skUl area^ a single result is given that 
shows the performance by all pupils in PVS. The District Report is -^r^' 
divided into two parts show ing overal ^ district performances on benchmark 

ski lis and on individual items. The School Report, has two parts showing _ 

overall school performance on benchmark sk 1 1 1 s and on individual Items* 
No breakdown of results by instructional program is given for the 



District Report or School Report, Classroom, Reports have the same 
three-part structure as they do on the Profictancy Ski 1 Is f nventory. 

Reports for the Hi d- Year fnventory 

The Mid- Year Inventory has four oiajor reports with the same 
structure as the Beginning Sk!] Is Inventory* Hwever, like the Proficiency 
Skills Inventory, the MYI Reports feature a breakdown of resul ts by. instruc* 
tional programs^ In addition, the fiational j School , and District Reports 
also show how much of each instructional program has been comp leted by 
mid-year. For example^ Part 1 of the District Report , showr ng results 
on benchmark skill areas, also shows the number of pupils that took each 
section of the Mid-Year Inventory, Since teachers administer only those 
sections of the Inventory that correspond to what has been completed by 
mid-year, the number of pupils that take each section of the inventory 
provides administrators with an estimate of how much of the program has 
been completed across the district. 

Reports on Learning Mastery Profiles 

No regular PVS reports are given on results generated by the Learning 



Mastery Prof 1 les s ince this component is used enti rely wi 
and no data are forwarded for addi tional processing. 

GENERAL OPERATIONS AND PROCEDURES 



thin the classroom. 



PVS has been designed and modi f led a number of times in order to 
create a system that wi 11 generate a maximum amount of informst Ion about 
ski 1 1 deve 1 opment for a given Investment In teacher and pupil time. The 
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major Inventory components in the nucleus of the PVS network are the 
necessary working un i ts of a system that can provide teachers and 
administrators with the information they have said is needed to maintain 
effectivaness in the development of mathemat I cs^-re lated skills^ While . 
any information system requires an investment of time from teachers and 
pupilSi PVS inventories are designed for an effective balance between 
processing costs to users and the amount of time required for administra^ 
tion and preparation for data process Ing . I n pract i ce, the use of PVS is 
little more complex than ordering components^ return Ing data to SWRL for 
processing, and receiving reports, 

Inittallzlnq Data Processing 

Designation of District PVS Coordinator ^ At the time initial orders 
of PVS components and/or reports are submitted, a PVS coordinator Is 
designated by the local school dl str I ct to receive and d Istribute al 1 
PVS materiaISi Including reports, and to set up procedures for processing 
of test results. These procedures Insure maximum compat Ibl 1 ity of district 
use of material swi th the system of procedures al ready established In PVS 
for efficient management of computer process i ng of results and the return 
of reports to appropriate district personnel* 

Ass I ^nment of Process Ing Codes . All PVS data processing is done by 
Identification numbers given to each d I str 1 ct , school , and teacher/cl assroom 
that is going to subml t mater I als for processing during the year. SVVRL , 
assigns a number to each district, and the PVS coordinator, m turn, assigns . ^ 
numbers to each school and classroom. The procedures for ass igning school 
and classroom numbers are quite simple. 



The district coordinator receives two types of forms for In 1 1 lal Urng 

the district data base. The first form, cal led the Distrjet Coordinator's 

School Identification Sheet, Is used by the PVS coordinator to assign 

schools to a set of 3-diglt codas that are pre-printed on the sheet. 

The sheet itself is actual ly a V'sandwich" of four self-carboning pages 

wi th spaces for fj fty schools. When school codes are ass Igned ^ the top 

copy is returned to SWRL at least one week prior to the time inventories 

will be submitted for data processing. This sheet Is used to check school 

and distri ct codes on sets of inventories from each classroOTi as they are 

sent to SWRL for process ing» AddltionSideletionSj and other changes in 

the list of school codes can be made by the PVS coordinator at any time 

during the year as long as a copy of such changes is sent to SWRL at 

least one week prior to the time inventor ies are sent for processingV 

The important thing is f or P\/S coord I nators to keep an up-to-date record 

of school codes, since the routing of Information to teachers depends. 

entirely upon accurate school and teacher information,: 

-■ , 
The second type of form that the coordinator receives Is the PVS 

PrincipaPs Teacher I dent I f i ca 1 1 on Sheet. Fifty or more copies of this 

form accompanies the PVS Coordinator's School Identification Sheet. On 

each sheet, the coordinator writes the school name and the same Identifying 

number that was ass i gned to the school on the PVS Coordinator's Tdentlficai- 

tion Sheet (the district number has already been pre'='pr f nted on each of 

the fifty sheets) . Each of; the Principal ' s Teacher Identification Sheets 

I s t hWn f on^aFdeJ ta 

designee can assign a 2^diglt ident i f Ication code for each teacher who 
is going to submit tests to Sl\fRL for processing. Teachers use this code 



on a 1 1 matBrla Is reLurned to SWRL. The prfncl pal's sheet Fs attached to 
a pre^addressed ma i 1 1 ng card con tai n ing a carbon copy of tlio school nanie 
and code and the district name and code j but no teacher names or codes. 
The principal or designee removes the card and returns it directly to 
SWRL as soon as the sheet is received from the PVS coordinatorv Receipt 
of this card confirms the schpoPs participation in PVS for the coming 
year. SWRL royt i ne ly checks the school name and number against its copy 
of the PVS Coord inator' s School I den t i f i cat i on Sheet to ensure that these 
codes match* No teacher names or codes need to be forwarded to SWRL. 
This wayj classrooni reports are identified only by teacher codes* This 
procedure p 1 aces respons I b i 1 1 ty on each school to maintain the list of 
teacher codes in order to channel each PVS c lassroom report to the appro^ 
priate teacher* The list of teacher codes cans of course, be changed by 
the school at any time, and It will not affect SWRL processing of results 
or the generation of PVS reports* This characteristic of PVS provides 
maximum f lex i b i 1 i ty at the school level for adapting use of PVS to changes 
in teacher assignments and in school and classroom organization during the 
school year, as long as a record of these adjustments is maintained at tfie 
school level so that a 1 1 teachers receive the appropriate PVS classroom 
report dealing with their students i 



In general, no other initial izat 



on procedures are requi red for use 



of regular PVS components. The Learning Mastery Profiles are used entirely 
within the classroom, wl th no data forwarded out of the classroom to 
district administrators or SWRL. 
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Report I n i t J a IXzat j^qn 

Receipt of reports for the Beginning Ski ] ] s I nventory , MId-Ysar 
Inventory, and Proficiency Skills Inventory requires no special 
initialization procedures. Reports at the national, district, school 
and classroom levels are generated automatically for the populations 
partScipating in each particular inventory, 

Adml n i s te r i nj I n ven tori es and 0 1 he r ComponentS- 

Geno raj P rocedu res^ . Administration of regular PVS Inventories 
(BSI, MYI and PS I ) is straightforward and simple. These inventories 
are group-administeredi and al 1 items are mul tiple-cho! ce. The PSI is 
ordinarily administered during the fi rst two weeks In May to allow time 
for return of reports to teachers before school is out. The BSI is 
ord i nari ly admi n i stered dur Ing the last two weeks in September and the 
MYI during the last two weeks in January* Levels 4^6 of al 1 PVS inven- 
tories, and the end-of-year inventory for Level 3f have separate answer 
sheets for pupils to mark responses* For i dent i f I cat! on , pupils mark 
their first and last Initials, A space is provided for teachers to give 
each pupil an optional i dent I f I cat i on nutiber , which they may want to use 
In cases where two pup! 1 s have the same initials. Pupils also mark their 
grade level (or some simi lar desi gnat Ion used In the school organization) , 
test level, and, on the PSI they mark the form (A, C, D) they are taking. 
Pupils mark answers to Items directly on the booklet for all inventories at 
Levels 1 and 2, and for the Beginning and Mid=Year Inventories at Level 3- 
They also /nark their initials (first and last) and grade levels but do not ^ 
mark the form or level since this information has been pre-coded on the 
booklet/ ; vv 24 7^ 
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SWRL assumes no respons i b M I ty for I nformat i on that is not recorded 
on classroom materials or not coded correctly. Teachers are advised to 
quickly scan booklets or answer sheets to see that information has been 
properly coded. 

Matching Levels of PVS Inventories to Levels of Program Materials . 
Teachers match levels of PVS inventories to the level of program materials 
that pupils are using in regular instruction. It Is common practice for 
instructional groups and entire classrooms to use Instruct ional mater ial s 

from a part i cul ar program that are "below convent lonal grade level In 
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such casesi teachers use the level of PVS inventory that matches the level 
of materials pupils are usingj so the inventory level that pupils mark on 
the I r answer sheets di ffers from thei r grade level in school • By assigning 
Inventdries in thi s way ^ pup i 1 s a re credited for having achieved a higher 
grade level J even though they are currently working out of materials 
intended for a lower grade level in the conventFona] school organ izat ion. 
Pupils who are using program mater 1 al s in this special way ( 1 .e. ^ grade 
levels Jn school do not match mventory levels) have their inventory results 
summarized at the classroom lewel in the same way as other pupils in the 
class. But on school and district reports T these special inventory results 
are summarized In a separate col umn on the report* In such. cases results 
are divided according to regular use of programs and PVS inventories (grade 
level and PVS inventory level match) and special use of programs and PVS 
inventories (grade level and PVS inventory leveldo not match). 

Special Procedure for the Mid-Year Inventory . The Mid-Year Inventory 
has an extra procedure fn admin i st rat ion because of 1 ts des Ign , I tems in 
this inventory are grouped by module according to "steps" in the development 
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of benchmark sk i 1 1 s , and a 1 1 benchmark skills for the year are included. 
In this way, PVS does not penalize teachers who do not complete instruction 
in these textbooks in a linear sequence. Before this inventory js 
administered, the teacher must identify those modules on the Inventory 
that represent the instruct i on that has actual 1 y been completed dur Ing 
the first half of the school year. 

Procedures for Use of the Learning Mastery Prof i 1 es . Since the 
Learning Mastery Profile is not a forma 1 assessment COTponentj It is 
used in a different way during the school year. There are two parts to 
the system: (1) a set of Skills Checks to be performed by the teacher In 
routine "grading" of reguVar practice from the pupiPs textj and (2) a set 
of reeord^keeping materials for organizing the information about pupils' 
progress. The Skills Checks are designed to accompany instruction in a 
speGific mathematics program i d! fferent instructional programs have different 
SWRL-deslgned^Learnlng Mastery Profiles, ' 

In use* the teacher refers to the User's Guide for fdentification of 
sped fi c problems in regular pupil mater i a 1 s wh I ch can be spot checked In 
practice activities, In order to obtain continuous feedback on how well 
each pupil is proq ress 1 ng . A Sk i 1 1 s Check sel dom amounts to more than 
three or four probl ems froni any given activity, and not more than one or 
two act I VI t i es des i gna ted as Ski ] 1 s Checks wou Id occur In a typ! ca 1 week 
of instruction. Of course, teachers may check more of pupils* work than 
just the problems des ignated as Ski 1 I s Checks, but these problems should 
be suf f icient to make up a profile of pupil progress. As Sk i 1 1 s-Check 
problems are examined In pupils' vjork, the teacher notes results on the 



special Profile sheet providF,d for each pupil. Opposite each Skills 
Cheekj the teaGher makes an informa] Judgment of how well a pup I 1 has 
done by circling a + , i/, or , to denote a ''good,*' ''adequate,'' or 
"less than adequate'' performance. Skills Checks are grouped according 
to major skill arenas involved in the structure of PVS inventory components, 
so that results of these "checks" on each profile sheet show the progress 
each pupil has made in each skill area. 

During the year, as pup i 1 s compl ete units or textbook activities, the 
teacher can record results of chapter tests frm the regular program In 
spaces provided on the Learning Hastery Profile sheet. In addition, scores 
on benchmark skill areas of the Beginning Skills Inventory^ the Mld^Year 
Inventory, and the Proficiency Ski 1 1 s Inventory can a I so be recorded in 

special spaces provided on the Profile sheet. In this way/ a completa 
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. record^of^_each child 's -pernor 

Sending Pupil Hater lals for Scoring and Processing 

Submitting PVS Components , The procedures for returning results for 
scoring and data processing are the same for all regular PVS inventories* 
As soon as adminrstratlon Is completed, teachers are advised to take a 
few minutes to chec!? pupil coding of f I rst and last Initials, PVS inventor/ 
level, and grade level* They are also advised to generally check pupils' 
marking of answers , to assure proper machlne'-scoring , since SWRL does not 
assume resppns Iblli ty for "clean up" of stray marks, poorly marked answers, 
or incomplete erasures. Teachers may assign a number to each pupil (In 
addition to first and last Initial) if desired* A place is provided for a 
supplementary pupl I-number on the booklet (levels N3) or answer jheet 



(levels k-G)\ Such a number is often useful i n d i f ferent I at ing between 
pupils with the same initials, and for other record keeping such as 
coordination of reports with pupil numbers in the teacher* s regular 
record book. All other PVS components are sent to SWRL using special 
procedures arranged wi th dl str ict. 

Completion of Classroom Identification Sheets . Each set of Inventory 
materials returned to SWRL is Identified by a Classroom Identification Sheet. 
This sheet has spaces for marking the district^ school, and classroom codes, 
and a special code for the Instructional program in use in the classroom. 
In instances where a teacher uses ent i rel y different programs with different 
Instructional groups, separate Classroom Identification Sheets can be marked 
(with different program codes) for each set of inventories. 

# 

The inventories and Identification Sheets are returned to the school 
pr ihcT^r who rttal IV SWftL at the same t ime* The" 

principal has responsibility for seeing that all inventories are admi nl stered 
and results returned to SWRL in time for processing. Under normal conditions 
the Beginning Skills Inventory is scheduled to arrive at SWRL by October 1^ 
the Mid-Year Inventory by February 1, and the Proficiency Skills Inventory 
by May 1 5 * 

Distribution of PVS Reports / ^ 

In most cases, PVS classroom reports are returned within 5 to 10 days 
of their receipt at SWRL* School and district reports are returned as soon 
as ail inventories have been received at SWRL. In any case, schools and 
districts receive al 1 reports from the Beginning Skills Inventory by about 
October 20, the Mid-Year Inventory by February 20,, and the Proficiency Skills 
Ski 1 Is Inventory by June 15 » 



-26- 

Districts have considerable flaxibilfty in deciding how reports are 
to be recefved and distributed. They may designate whether teacher and 
principal reports are to be returned to the PVS coordinator or directly 
to the principals and teachers. Under regular procedures, all reports 
are sent to the PVS coordinator* Classroom reports are all enclosed in 
separate envelopes , packaged by school, together with the school report. 
Each report Is sealed, even though the PVS coordinator distributes school 
packages to principals, who, in turn, distribute classroom reports to 
teachers. In this way, each member of the district personnel receives 
information specifically pertaining to their level of interest, unless 
special j n format lon-'hand 1 1 ng arrangements are made among admin istrators , 
principals, and teachers. 

An arrangement can be made where all classroom and school reports are 
ma I led'dl rectly to teachers and principals* By request (and at add 1 1 lona 1 ^ 
cost), the PVS coordinator can receive a copy of all school and classroom 
reports, although principals and teachers will be notified that response 
has been made to such a requesti Under special arrangements, all reports 
are distributed in the same way. If any principals and teachers in a 
district receive reports directly, then all must receive them in the same 
way. If district coordinators wish to have copies of any school and class-" 
room reports, then copies of all school and classroom reports will be sent 
to the d i St r I ct coord i na tor , Similar procedures apply for school principals 
who may want their own copy of each classroom report* 
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DEVELOPING A LOCAL PROFICIENCY VERIFiCATION SYSTEM 

Proficiency Verfficatlon Systems ts 'actually a large network of 
assessmant and management components s reports, and other surveys and 
servicei which are available for local districts to develop informatidn 
systems to meet their own needs. SWRL's position as a research and 
development Institution allows for districts to develop a cooperative 
relationship with SWRL in select ing components and services, but also 
In designing new ones that do not presently exist. 

Selection of PVS Components ^ PVS components can be selected by 
school districts In patterns that best fit local needs for information 
about proficiency development. Information generated by PVS components 
can be used In a variety of ways including long-range plann ing , program 
accompl i shmentSi and Instructional management by individual teachers from 
wee k-tO" week. 

Adapting PVS Reports to Local Information Needs * A local system can 
phase-In the amount of Information It receives with the ability of the 
dlstrjct organiHation to use large quantltjes of highly detailed data at 
several points during the school year. Through in-service training on the 
interpretation and application of specialised data, local school organisa- 
tions can continually increase their capacity for evaluating and improving 
the quality of Instruction which they provide. 

PVS can supply an unprecedented amount of Information for reasonably 
modest Investments of time from admin Istratorsj teachersi and pupils. 



Developing Local Channels of rnformation Flovv . Distribution options 
for PVS reports provide school districts with a un i que opportunl ty for 
cooperaLlve davelop<ont among adml n i s trat Ive and teaching personnel to 
share proficiency Information, In normal use, PVS coordinators do not 
have information on performances below the district level, while teachers 
will not have macro^summar I es above the level of their own classroom* 
Effective use of PVS information mandates cooperative use of proficiency 
information across several levels of school organization, all for the 
purpose of improving instruction and opportunities for children to learn 
needed skills. SWRL can best function in this arena by providing technical 
assistance in adjusting information on each report, and by adapting the 
routing of PVS reports to patterns of Information flow developed by 
personnel from all levels of the administrative structure* 

RESEARCH AND DEVELOPMENT 

SWRL maintains a continuous program of research and development in 
PVS and in other areas of education information systems as well. Districts 
who use PVS have the opportunity to not only Implement new developments 
as soon as they are available, but to participate in such development 
through cooperative planning* in order to meet local needs. 
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